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1.  Background 


System  designers  at  the  U.S.  Army  Tank- Automotive  and  Armaments  Command  (TACOM) 
have  developed  a  crew  integration  and  automation  test  bed  (CAT)  advanced  technology 
demonstrator  (ATD).  TACOM  system  designers  use  the  CAT  ATD  to  demonstrate  the  crew 
interfaces,  automation,  and  integration  technologies  required  to  operate  and  support  future 
combat  vehicles.  One  of  the  goals  of  the  CAT  ATD  program  is  to  use  advanced  technologies 
such  as  speech  recognition,  three-dimensional  (3-D)  audio,  indirect  vision,  etc.,  to  decrease  crew 
workload  and  allow  the  crew  more  time  to  perform  their  mission. 

The  CAT  ATD  consists  of  two  identical  advanced  technology  crew  stations,  along  with  a  safety 
driver  crew  station,  integrated  into  a  modified  Bradley  fighting  vehicle  chassis.  Two  crew 
stations  are  included  in  the  CAT  ATD  because  the  CAT  ATD  program  assumes  that  the  future 
combat  vehicle  will  be  a  two-person  crew.  The  two  identical  crew  stations  are  called  the 
Vetronics  Technologies  Test  Bed  (VTT)  station  soldier-machine  interface  (SMI)  simulator. 

TACOM  system  designers  can  use  the  VTT  SMI  to  determine  if  crew  members  can  drive  while 
they  are  also  performing  other  mission-related  tasks,  such  as  engaging  targets,  scanning  for 
targets,  and  command  and  control.  Researchers  can  also  use  the  VTT  to  evaluate  different 
configurations  and  types  of  displays  and  controls.  For  example,  they  might  compare  the 
performance  of  a  crew  member  who  is  sending  command  and  control  (C2)  messages  by  pushing 
buttons  on  a  flat  panel  display  to  the  performance  of  that  same  crew  member  who  is  using  verbal 
commands  to  send  C2  messages.  To  make  these  comparisons,  the  equipment  in  the  VTT  would 
have  to  be  reconfigured.  Furthermore,  to  ensure  that  the  results  of  their  comparisons  are  valid, 
the  system  designers  would  have  to  test  a  large  number  of  crew  members  operating  the  VTT 
SMI.  However,  reconfiguring  the  VTT  SMI  equipment  and  obtaining  an  adequate  number  of 
crew  members  to  test  would  be  expensive.  Therefore,  TACOM  system  designers  asked  U.S. 
Army  Research  Laboratory  (ARL)  researchers  to  build  human  performance  models  to  represent 
the  performance  of  the  crew  members  operating  the  VTT  SMI. 

TACOM  provided  the  ARL  human  performance  models  to  General  Dynamics  Land  Systems 
(GDLS)  engineers,  who  are  the  contractors  responsible  for  the  next  phase  of  the  CAT  ATD. 
GDLS  engineers  can  modify  these  models  or  build  their  own  similar  models  to  represent  several 
different  configurations  of  equipment  and  function  allocations.  Multiple  runs  of  the  ARL  models 
should  be  useful  in  predicting  the  optimum  configuration  of  VTT  SMI  equipment  and  function 
allocation  between  crew  members.  The  GDLS  engineers  would  then  use  the  optimum 
configuration  of  equipment  and  allocation  of  functions  predicted  by  the  models  to  evaluate  crew 
member  performance  of  mission  functions  in  the  VTT.  This  evaluation  process  allows  the 
contractor  to  evaluate  more  configurations  of  equipment  in  less  time  with  fewer  crew  members 
and  is  therefore  more  effective  and  less  expensive  for  TACOM. 
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2.  Model  Development 


2.1  VTT  SMI  Description 


The  VTT  consists  of  two  crew  stations.  Each  crew  station  contains  the  same  equipment:  a  seat, 
a  yoke,  two  pedals,  keyboard,  and  six  computer  displays.  The  crew  station  layout  is  shown  in 
Figure  1.  The  crew  station  operator  functions  as  a  commander  and  driver  or  gunner  and  driver. 

If  both  crew  stations  are  operating,  one  crew  member  is  the  gunner,  the  other  crew  member  is  the 
commander,  and  either  crew  member  could  be  the  driver.  In  addition,  the  crew  members  swap 
the  driving  tasks  if  necessary. 


Figure  1.  Vetronics  technology  test  bed  station  SMI. 


A  crew  member  simulates  driving  the  VTT  using  an  indirect  driving  system.  This  indirect 
driving  system  allows  the  crew  member  driving  in  the  VTT  to  see  outside  the  vehicle  through  the 
use  of  video  imagery.  This  video  imagery  is  displayed  for  the  crew  member  on  three  of  the  six 
computer  displays  that  are  situated  horizontally  in  each  crew  station.  The  remaining  three 
displays  in  the  crew  station  are  multi-functional  displays.  These  displays  are  arranged 
horizontally  and  situated  just  below  the  indirect  vision  displays.  The  crew  member  selects  any 
one  of  these  multi-functional  displays  to  show  the  instrument  panel  needed  for  simulated  driving 
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of  the  vehicle.  Each  crew  station  is  also  equipped  with  a  pedal  and  yoke  to  simulate  steering, 
accelerating,  and  decelerating  of  the  vehicle. 

In  addition  to  a  simulated  driving  capability,  the  VTT  is  equipped  with  a  simulated  target 
acquisition  capability  that  allows  the  operator  to  search,  detect,  recognize,  identify,  “lock  on,” 
and  track  computer-generated  targets.  If  two  crew  members  are  operating  in  the  VTT,  the  target 
acquisition  system  permits  each  crew  member  to  search  independently  for  the  targets.  To  show 
the  target  acquisition  information,  the  crew  member  chooses  any  one  of  the  three  multi¬ 
functional  displays. 

In  addition  to  simulated  driving  and  target  acquisition  systems,  the  VTT  is  equipped  with  a  C2 
system,  situational  awareness  capability,  and  auditory  alerts.  The  C2  system  permits  digital  and 
voice  (intercom  and  radio)  message  communications.  The  situational  awareness  data  are 
battlefield  data  that  are  displayed  on  a  two-dimensional  (2-D)  color  map  or  with  a  3-D  battlefield 
visualization  capability.  The  C2  and  battlefield  visualization  information  can  be  displayed  on 
any  one  of  three  multi-functional  displays.  The  auditory  alerts  are  warnings,  cautions,  and  alert 
messages  to  the  crew  members  when  conditions  warrant,  i.e.,  a  digital  message  is  sent  or 
acknowledged.  The  crew  member  is  equipped  with  a  headset  intercom  system  that  allows  him  or 
her  to  hear  the  auditory  warnings.  This  intercom  system  also  permits  him  or  her  to  talk  to  the 
other  crew  member  and  send  radio  messages. 

TACOM  system  designers  use  the  capabilities  of  the  VTT  simulator  to  evaluate  the  crew 
interfaces,  automation,  and  integration  required  to  operate  and  support  future  combat  vehicles. 

As  part  of  the  evaluation,  TACOM  system  designers  developed  two  combat  scenarios  for  the 
simulator:  scenario  1,  a  tank  mission,  and  scenario  2,  a  scout  mission.  These  scenarios  present 
crew  members  in  the  simulator  with  a  flow  of  tactical  events  that  require  them  to  perform  the 
functions  and  tasks  they  would  perform  in  combat  during  a  tank  or  scout  mission.  As  the  crew 
members  perform  these  tasks,  the  TACOM  system  designers  observe  them  and  identify  potential 
human  factors  problems  with  the  controls,  displays,  and  allocation  of  tasks  between  crew 
members  (Smart,  Rapkoch  II,  Dahill,  Fritz,  and  Williams,  1997). 

One  critical  human  factors  area  evaluated  by  the  TACOM  system  designers  is  mental  workload. 
The  system  designers  want  the  mental  workload  to  be  evenly  distributed  and  manageable  as  the 
crew  members  perform  their  tasks.  To  help  them  predict  the  mental  workload  of  the  crew 
members  as  they  perform  their  tasks,  ARL  researchers  have  built  task-network  models  of  the 
VTT  simulator.  To  build  these  networks,  they  used  a  human  performance-modeling  tool  called 
IMPRINT. 

2.2  IMPRINT  Description 

IMPRINT  is  a  PC-based,  human  performance-modeling  tool.  System  designers  or  analysts  use 
IMPRINT  to  estimate  the  likely  performance  of  a  new  military  system  by  building  models  of 
each  operational  or  maintenance  mission  that  the  system  will  be  required  to  accomplish 
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(Micro Analysis  and  Design,  1998).  Because  one  of  the  critical  components  of  operator  and 
system  performance  is  operator  workload,  IMPRINT  has  two  options  for  generating  workload 
profiles:  visual,  auditory,  cognitive,  and  psychomotor  workload  option  (VACP)  and  an 
advanced  workload  option.  ARL  researchers  used  the  advanced  workload  option  for  the  VTT 
modeling  project.  The  underlying  structure  for  the  advanced  workload  option,  as  well  as 
IMPRINT  itself,  is  task  network  modeling  and  discrete  event  simulation. 

Task  network  models  are  computer-based  simulation  models  that  the  system  designer  can  use  to 
predict  task  times,  task  accuracy,  and  mental  workload.  The  model  consists  of  the  tasks  needed 
to  accomplish  a  particular  job  or  mission,  the  operator  who  performs  each  task,  the  amount  of 
time  it  takes  each  task  to  execute,  probability  of  success  and  the  sequence  by  which  the  tasks  are 
performed. 

To  build  a  task  network  model,  the  system  designer  begins  with  a  mission  or  job  that  an 
operator(s)  will  be  performing.  Next,  the  system  designer  reduces  the  mission  to  the  specific 
tasks  and  subtasks  that  the  operator(s)  need  to  accomplish  to  perform  the  mission.  He  or  she 
then  enters  the  sequence  by  which  the  operator(s)  perform  the  tasks.  This  process  is  relatively 
easy  for  the  designer  to  perform  and  can  be  done  early  in  the  design  process.  It  has  the  added 
benefit  of  forcing  the  designer  to  think  in  great  detail  about  all  of  the  proposed  system.  In 
addition  to  the  tasks  to  be  performed  and  their  sequence,  the  system  designer  building  a  task 
network  model  must  also  estimate  the  time  required  for  the  operator  to  complete  each  task. 

These  task  times  are  then  entered  into  the  computer  model  and  used  to  calculate  overall  system 
performance  time.  If  the  task  network  model  is  a  deterministic  model  with  non-parallel  tasks, 
then  the  times  entered  for  each  task  will  sum  to  the  estimated  mission  time.  With  a  stochastic 
task  network  model,  however,  the  times  for  each  task  can  be  drawn  from  a  distribution  of  times. 
The  resulting  system  performance  times  for  the  stochastic  model  are  an  average  of  the  randomly 
selected  times.  Because  human  behavior  varies,  the  stochastic  task  network  model  is  a  more 
realistic  representation  of  system  operators  than  the  deterministic  model.  Therefore,  the 
IMPRINT  software  is  based  on  a  stochastic  rather  than  deterministic  task  network  modeling 
technique. 

In  addition  to  being  a  stochastic  task  network  modeling  tool,  IMPRINT  uses  discrete  event 
simulation  to  model  human  performance.  Discrete  simulation  and  continuous  simulation  are  two 
broad  classes  of  simulation  models.  The  type  of  simulation  an  analyst  uses  depends  upon  the 
types  of  problems  to  be  solved.  “For  example,  a  model  of  traffic  flow  on  a  freeway  would  be 
discrete  if  the  characteristics  of  and  movement  of  individual  cars  are  important.  Alternatively,  if 
the  cars  can  be  treated  fin  the  aggregate,’  the  flow  of  traffic  can  be  described  by  differential 
equations  in  a  continuous  model”  (Law  and  Kelton,  1991).  Because  the  system  designer  is 
interested  in  the  operator’s  tasks  individually,  not  as  an  aggregate,  discrete  simulation  was 
selected  as  the  basis  for  IMPRINT.  More  specifically,  IMPRINT  is  based  on  a  discrete  event 
simulation.  A  discrete  event  simulation  is  appropriate  when  the  elements  of  the  process  modeled 
have  a  distinct  beginning  and  ending  as  do  all  the  missions  and  tasks  modeled  with  IMPRINT. 
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The  specific  task  network  tool  and  discrete  event  simulation  model  used  in  the  IMPRINT 
software  is  the  commercially  available  Micro  Saint. 

Micro  Saint  is  a  task  network  simulation  language  developed  by  MicroAnalysis  and  Design 
(MA&D)  for  the  U.S.  Army  Materiel  Research  and  Materiel  Command.  Micro  Saint  makes  it 
simple  for  system  designers  to  build  task  network  models  by  providing  a  graphical  user  interface 
(GUI)  and  flow  chart  approach  to  modeling.  Using  Micro  Saint,  the  system  designer  builds  a 
graphical  representation  of  the  task  network  that  is  also  a  graphical  representation  of  the  job  or 
mission  that  the  system  operator  will  be  performing.  Figure  2  depicts  an  example  of  a  graphical 
representation  of  a  task  network. 
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Figure  2.  An  example  of  a  task  network. 


As  Figure  2  depicts,  the  system  designer  uses  rounded  rectangles  to  represent  modeled  tasks  and 
arrows  to  connect  the  tasks  to  represent  the  order  of  task  execution  within  the  mission.  Once  the 
designer  has  completed  the  task  network,  the  Micro  Saint  software  links  the  tasks,  task  times, 
and  individual  performance  together  in  the  model  and  simulates  the  system  operator  performing 
a  mission.  The  system  designer  uses  a  modified  version  of  the  Micro  Saint  GUI  to  model  the 
operator’s  tasks  with  IMPRINT.  In  addition  to  modeling  the  tasks  and  task  times,  however,  the 
system  designer  can  also  use  IMPRINT  to  model  operator  mental  workload  with  either  of  its  two 
options,  VACP  or  advanced  workload  analysis.  The  advanced  workload  analysis  was  selected 
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for  this  project  because  the  workload  estimate  generated  from  this  option  allows  the  analyst  to 
consider  what  particular  equipment  the  operator  is  using  to  do  his  or  her  tasks. 

2.3  Advanced  Workload  Analysis 

The  relationship  between  workload  and  performance  is  complex.  It  is  not  simply  that  as 
workload  increases,  performance  decreases.  Instead,  the  relationship  between  workload  and 
performance  is  traditionally  described  as  an  inverted  “U”  relationship  because  decrements  in 
performance  may  occur  if  workload  is  either  too  low  or  too  high.  Furthermore,  there  can  be  a 
disassociation  between  workload  and  performance  at  certain  levels.  This  means  that  as  workload 
increases,  the  operator’s  performance  may  not  decrease  because  the  operator  has  a  strategy  for 
handling  task  demands  to  compensate  for  the  increased  workload.  Hart  (1989)  proposed  that 
operator  workload  strategies  play  an  important  role  “in  determining  the  relationship  between 
objective  task  demands,  experienced  workload,  and  system  performance  (p.4).”  The  advanced 
workload  analysis  feature  of  IMPRINT  allows  the  system  designer  to  incorporate  operator 
workload  management  strategies  into  the  workload  model. 

To  obtain  a  workload  estimate  using  the  advanced  workload  analysis  option,  the  system  designer 
first  selects  a  job  or  mission  that  the  operator(s)  of  the  proposed  system  will  perform.  Next,  the 
system  designer  segregates  the  selected  mission  into  the  tasks  the  operator(s)  will  be  performing 
in  order  to  accomplish  this  mission.  The  task  sequence  and  the  operator(s)  who  will  perform 
these  tasks  are  then  identified.  Next,  the  advanced  workload  analysis  option  links  workload  to  the 
specific  equipment  used  by  the  operator.  This  option  links  workload  and  equipment  because  it 
contains  an  embedded  workload  calculation  algorithm.  This  algorithm,  which  is  depicted  in 
Figure  3,  is  based  on  a  variation  of  the  W/INDEX  model  (North  and  Riley,  1989).  It  calculates 
workload  based  on  the  resources  being  used  by  the  operator  and  incorporates  the  fact  that  multiple 
tasks  are  being  performed  simultaneously.  In  addition,  the  algorithm  relates  the  resources  used  to 
crew  station  displays  and  control  surfaces  (Little  et  al.,  1993).  Because  the  advanced  workload 
analysis  option  contains  this  algorithm,  the  system  designers  must  specify  the  equipment 
interfaces  (e.g.,  keyboard,  helmet-mounted  display)  that  operators  will  be  using  to  accomplish 
each  of  their  assigned  tasks.  Furthermore,  the  designer  must  also  specify  the  mental  resources 
that  a  crew  member  uses  with  each  equipment  interface  as  he  or  she  performs  each  task. 

With  the  advanced  workload,  method,  the  mental  resources  are  a  set  of  five  channels:  visual, 
auditory,  cognitive,  psychomotor,  and  speech.  In  addition,  the  system  designers  can  create  their 
own  resources.  For  example,  the  system  designers’  research  may  indicate  that  the  tactile 
resource  is  important  for  their  design.  They  can  then  add  this  to  the  resource  list  in  the  advanced 
workload  analysis  option.  However,  no  default  scales  are  available  to  help  the  designers 
estimate  workload  for  this  resource.  They  must  substantiate  values  for  this  resource,  based  on 
current  research.  If  the  system  designers  choose  to  use  the  default  resources,  they  can  rate  the 
amount  of  each  of  these  resources  required  to  do  a  task  with  7-point  rating  scales.  These  scales 
are  modified  versions  of  scales  developed  by  McCracken  and  Aldrich  (1984).  The  McCracken 


6 


and  Aldrich  scales  have  been  revised  in  the  advanced  workload  analysis  option  because  the 
original  scales  were  developed  for  estimating  workload  for  the  Army’s  light  helicopter.  Based 
on  subject  matter  experts’  (SMEs’)  recommendations,  some  of  the  scale  values  were  revised  to 
represent  workload  for  Army  tanks.  In  addition,  the  psychomotor  resource  was  divided  into  two 
separate  resources:  motor  and  speech.  The  revised  scales  are  provided  in  Table  1.  The  system 
designers  use  these  scales  to  estimate  the  resources  required  for  each  task  an  operator  performs. 
After  the  system  designer  has  entered  the  workload  values,  the  workload  algorithm  embedded  in 
IMPRINT  calculates  the  mental  workload.  The  calculation  method  in  this  algorithm  is  based  on 
multiple  resource  theory  (MRT). 


Wt  =  instantaneous  workload  at  time  T 
i,  j  =  1...1  are  the  interface  channels 
t  =  1 . .  .m  are  the  operator’s  tasks  or  activities 
a  sj  =  attention  to  channel  i  required  to  perform  simultaneous  task  s 
a  t,i  =  attention  to  channel  i  required  to  perform  task  t 
Cjj  =  conflict  between  channels  i  and  j 
Cjj=  conflict  within  channel  I 
and 

1.  if  a^i  or  asj  =  0,  then  (a,j  +  asj  =0); 

2.  if  atj  or  a^p  0,  then  (atj  +  asj  =  0). 

Figure  3.  Workload  algorithm. 

According  to  MRT,  when  an  individual  performs  a  task,  he  or  she  requires  different  mental 
operations  and  to  some  extent,  each  operation  uses  the  mental  processing  resources  necessary  to 
accomplish  the  task.  These  mental  resources  are  limited  and  a  supply-and-demand  problem 
occurs  when  the  individual  performs  two  or  more  tasks  that  require  a  single  resource.  As  a  result 
of  time  sharing  of  resources,  some  task  performance  times  may  increase,  probability  of 
successfully  completing  a  task  may  change,  or  performance  times  may  decrease  (Little  et  al., 
1993).  These  MRT  concepts  are  the  underlying  assumptions  for  the  advanced  workload  option 
in  IMPRINT.  MRT  theory,  however,  also  explains  how  two  tasks  can  conflict  with  each  other. 

According  to  the  multiple  resource  model,  two  concurrent  tasks  will  suffer  greater  interference  to 
the  extent  that  the  component  tasks  are  more  difficult  (demand  more  resources)  and  that  the 
components  compete  for  overlapping  resources.  Furthermore,  the  effects  of  difficulty  and 
resource  overlap  interact.  The  greater  the  degree  of  resource  overlap,  the  more  pronounced  will 


be  the  effect  of  the  level  of  difficulty  of  one  task  on  the  level  of  performance  of  another  task 
(Little  et  al.,  p  9).  The  workload  algorithm  in  the  advanced  workload  analysis  option 
incorporates  the  MRT  findings.  It  sums  the  resource  demands  and  also  includes  penalties  for 
situations  when  two  tasks  require  the  same  resources  and  for  situations  when  the  use  of  different 
resources  causes  interference.  The  workload  algorithm  itself  is  presented  in  Figure  3. 


Table  1.  Revised  UH-60  helicopter  workload  component  scales 


Scale  Value 

Descriptors 

New  Values 

Visually  unaided  (naked  eye) 

1.0 

Visually  register/detect  (detect  occurrence  of  image) 

3.0 

Y  3.7 

Visually  discriminate  (detect  visual  differences) 

5.0 

4.0 

Visually  inspect/check  (discrete  inspection/static  condition) 

3.0 

5.0 

Visually  locate/align  (selective  orientation) 

4.0 

5.4 

j  Visually  track/follow  (maintain  orientation) 

4.4 

5.9 

Visually  read  (symbol) 

5.0 

7.0 

Visually  scan/search  monitor  (continuous/serial  inspection,  multiple  conditions) 

6.0 

5.0 

Visually  aided  (night  vision  goggles  fNVGl) 

4.0 

Visually  register/detect  (detect  occurrence  of  image)  with  NVG 

5.0 

4.8 

Visually  inspect/check  (discrete  inspection/static  condition  (with  NVG) 

5.0  “j 

5.0 

Visually  discriminate  (detect  visual  differences)  with  NVG 

r  7.0  ~j 

5.6 

Visually  locate/align  (selective  orientation)  with  NVG 

5.0 

6.4 

Visually  track/follow  (maintain  orientation)  with  NVG 

5.4 

7.0 

Visually  scan/search/monitor  (continuous/serial  multiple  conditions)  with  NVG 

7.0 

Auditory 

1.0 

Detect/register  sound  (detect  occurrence  of  sound) 

1.0 

2.0 

Orient  to  sound  (general  orientation/attention) 

2.0 

4.2 

Orient  to  sound  (selective  orientation/attention) 

4.2 

HIHI 

Verify  auditory  feedback  (detect  occurrence  of  anticipated  sound) 

4.3 

HHI 

Interpret  semantic  content  (speech)  simple 

■ 

(1  to  2  words)  complex  sentences 

HUH 

6.6 

Discriminate  sound  characteristics  (detect  auditory  difference) 

®  7.0 

Interpret  sound  patterns  (pulse  rates,  etc.)  “j 

7.0 

Cognitive 

Automatic  (simple  association) 

1.2 

1.2 

Alternative  selection 

1.2 

l  3.7 

Sign/Signal  recognition  “ 1 

3.7 

4.6 

Evaluation/judgment  (consider  single  aspect)  1 

4.6 

5.3 

Encoding/decoding,  recall 

5.3 

6.8 

Evaluation/judgment 

6.8 

7.0 

Estimation,  calculation,  conversion 

6  8 

Rehearsal 

5.0 

Psychomotor 

(this  scale  was  divided  into  speech  and  motor  in  revised  scale 

1.0 

Speech 

Speech  simple  (1  to  2  words) 

II 

complex  (sentence) 

■■ 

Motor 

2.2 

Discrete  actuation  (button,  toggle,  trigger) 

2.2 

2.6 

Continuous  adjustive  (flight  control,  sensor  control) 

2.6 

4.6 

Manipulative 

4.6 

5.8 

Discrete  adjustment  (rotary,  vertical  thumbwheel,  lever  position) 

5.5 

6.5 

Symbolic  production  (writing) 

6.5 

7.0 

Serial  discrete  manipulation  (keyboard  entries) 

7.0 
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The  first  part  of  the  workload  algorithm  computes  the  resource  demands  for  all  active  tasks. 
Therefore,  each  time  a  new  task  is  started,  the  algorithm  adds  the  workload  ratings  for  each 
resource  for  the  new  task  and  all  other  tasks  being  performed  at  that  time.  The  next  two  terms 
within  the  second  bracket  of  the  equation  compute  the  penalties  for  two  tasks  using  the  same 
resource  at  the  same  time  and  two  tasks  requiring  different  resources  at  the  same  time.  For 
example,  if  one  task  requires  a  system  operator  to  look  at  a  computer  screen  on  the  right  side, 
while  a  second  task  occurring  simultaneously  requires  the  operator  to  look  at  a  computer  screen 
on  the  left  side,  then  the  equation  assigns  a  penalty  to  the  task.  In  this  case,  the  penalty  would  be 
that  one  task  could  not  be  performed.  In  other  cases,  the  penalty  might  be  that  a  task’s  time  is 
increased.  The  penalty  is  assigned  because  the  two  tasks  are  using  the  same  resource  at  the  same 
time.  The  system  designer  determines  the  amount  of  interference  between  resources  being  used 
by  concurrent  tasks  by  entering  values  into  a  conflict  matrix  provided  in  the  software.  The 
conflict  matrix  displays  each  resource  paired  with  the  equipment  interface  that  uses  the  resource. 
For  each  resource  and  interface  pair,  the  designers  enter  conflict  values  based  on  guidance  from 
their  own  research,  or  the  software  can  provide  default  values  based  on  the  MRT  literature.  The 
conflict  values  can  range  from  0  (represents  no  conflict)  to  1 .0  (represents  total  conflict).  The 
conflict  values  will  be  unique  to  each  system  design  because  the  values  are  linked  to  both 
equipment  interfaces  and  resources.  Each  design  will  have  a  different  set  of  equipment 
interfaces  that  use  specific  resources  and  therefore  its  own  set  of  conflict  values. 

After  the  system  designers  have  provided  conflict  values  and  workload  ratings  for  each  operator 
for  each  task,  the  algorithm  calculates  the  workload  for  each  operator  before  the  start  of  a  new 
task,  at  the  start  of  a  new  task,  and  finally  when  a  task  is  completed.  In  addition  to  calculating 
this  overall  workload,  the  advanced  workload  analysis  option  allows  system  designers  to  specify 
how  the  system  operator  will  manage  the  workload.  They  do  this  with  the  workload 
management  strategies.  These  strategies,  however,  were  not  used  in  the  VTT  modeling  effort. 

The  advanced  workload  analysis  reports  include  a  mission  summary,  critical  path  report, 
workload  graphs,  and  reports  describing  the  operator  activity,  overload,  and  channel  conflicts. 
From  these  reports,  system  designers  can  view  the  total  workload  value  over  time  for  each 
operator  of  the  system.  Because  this  value  is  on  an  ordinal  scale,  it  allows  the  system  designer  to 
make  only  relative  comparisons  of  workload  at  different  times  during  the  mission.  This  means 
that  the  system  designer  should  not  compare  specific  overall  workload  numbers.  Instead,  the 
designer  examines  the  workload  graphs  and  determines  where  the  workload  peaks  are  and  which 
tasks  were  operating  at  that  time  and  contributed  to  the  peaks.  The  designer  can  then  select  these 
tasks  as  candidates  for  redesign,  automation,  or  reallocation  to  another  crew  member  (Archer, 
1998).  Furthermore,  the  process  of  building  and  entering  data  into  the  models  helps  the  system 
designers  to  think  about  those  interfaces  and  tasks  that  are  contributing  factors  to  workload  and 
performance. 
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2.4  VTT  IMPRINT  models 


The  task  network  models  ARL  built  were  built  in  advanced  workload  IMPRINT.  They 
represented  two  crew  members  performing  the  functions  and  tasks  of  two  scenarios  provided  by 
TACOM.  Scenario  1  was  a  tank  mission  and  Scenario  2  was  a  scout  mission.  For  each  scenario, 
two  models  were  built  at  different  levels  of  task  detail.  Specifically,  one  model  included  tasks 
for  each  operator  button  push.  The  tasks  in  the  other  model  were  broader  tasks.  An  example  of 
a  task  in  the  first  model  might  be  “push  C2  button.”  The  task  in  the  second  model  might  be 
send  C2  message.  Examples  of  the  task  networks  for  each  model  are  provided  in  Figures  8 
and  9. 

ARL  researchers  decided  that  the  button-push  models  were  necessary  because  it  was  anticipated 
that  the  nature  of  some  tasks  might  change  in  future  versions  of  the  VTT.  Specifically,  some  of 
the  command  and  control  tasks  that  are  currently  performed  by  a  user  touching  a  flat  panel 
display  might  be  converted  to  voice-activated  commands.  Comparing  the  mental  workload  of 
these  two  sets  of  interfaces,  touch  versus  voice,  would  necessitate  a  model  built  to  the  button- 
push  task  level.  Therefore,  ARL  researchers  modified  the  initial  tank  and  scout  models  to  more 
detail  that  simulated  operations  as  simplistic  as  a  keystroke. 

The  initial  models  and  the  button-push  models  developed  by  ARL  consisted  of  two  sets  of  VTT 
models:  two  tank  models  and  two  scout  models  (see  Table  2).  The  VTT  tank  models  are  called 
VTT  tank  and  VTT  tank  Alexandria.  The  VTT  tank  Alexandria  model  is  detailed  to  the  button- 
push  level.  In  both  models,  ARL  built  an  advanced  IMPRINT  task  network  of  a  tank 
commander  and  a  gunner  performing  the  functions  and  tasks  of  TACOM  scenario  1  in  the  VTT 
SMI.  In  the  preliminary  simulation  runs,  the  tank  commander  performs  the  functions  of  driving, 
monitoring  communications  from  higher  headquarters,  scanning  for  targets,  communicating  with 
the  gunner.  Additionally,  the  commander  performs  all  command  tasks  associated  with  the  tank 
mission,  e.g.,  developing  the  fire  plan,  planning  routes,  etc.  The  gunner  performs  the  functions 
of  scanning  for  targets,  communicating  with  the  commander,  engaging  targets,  and  destroying 
targets. 

In  subsequent  runs,  the  driving  function  is  reallocated  to  the  gunner.  In  these  runs,  the  gunner 
drives  and  performs  the  functions  of  scanning  for  targets,  communicating  with  the  commander, 
engaging  targets,  and  destroying  targets  (see  Table  3). 

The  VTT  scout  models  are  similar  to  the  tank  models.  The  models  are  called  VTT  Scout  and 
VTT  Scout  Alexandria.  The  VTT  Scout  Alexandria  models  tasks  to  the  button-push  level.  In 
both  models,  ARL  built  an  advanced  IMPRINT  task  network  of  a  platoon  leader  and  a  sergeant 
performing  the  functions  and  tasks  of  TACOM  scenario  2  in  the  VTT  SMI.  In  the  preliminary 
simulation  runs,  the  platoon  leader  performs  the  functions  of  driving,  monitoring  communi¬ 
cations  from  higher  headquarters,  scanning  for  targets,  communicating  with  the  sergeant. 
Additionally,  the  platoon  leader  performs  all  reconnaissance  tasks  associated  with  the  scout 
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mission.  The  sergeant  performs  the  functions  of  scanning  for  targets,  communicating  with  the 
commander,  engaging  targets,  and  destroying  targets. 


Table  2.  Advanced  IMPRINT  models  of  the  VTT 


IMPRINT  Model 

Level  of  Task  Detail 

VTT  Tank 

no  button-  push  tasks 

VTT  Tank  Alexandria 

button-push  tasks  included 

VTT  Scout 

no  button-push  tasks 

VTT  Scout  Alexandria 

button-push  tasks  included 

In  subsequent  runs,  the  driving  function  is  reallocated  to  the  sergeant.  In  these  runs,  the  sergeant 
drives  and  performs  the  functions  of  scanning  for  targets,  communicating  with  the  commander, 
engaging  targets,  and  destroying  targets  (see  Table  3). 


Table  3.  Analysis  conditions 


Analysis 

Condition 

Model  Name 

Crew  Members 

Primary  Responsibility 

Driving 

Scanning 

1 

Scout  Alexandria 

■SB 

Platoon  Leader 

X 

X 

Sergeant 

X 

2 

Scout  Alexandria 

Platoon  Leader 

X 

Sergeant 

X 

X 

3 

Tank  Alexandria 

■BHHH 

Tank  Commander 

X 

X 

Gunner 

X 

4 

Tank  Alexandria 

Tank  Commander 

X 

Gunner 

X 

X 

In  addition  to  building  the  task  networks  for  the  scout  and  tank  models,  the  ARL  researchers  had 
to  incorporate  workload  data  into  the  models.  The  IMPRINT  workload  equation  is  presented  in 
Figure  3.  To  satisfy  the  requirements  of  this  IMPRINT  workload  equation,  the  ARL  researchers 
entered  into  the  IMPRINT  model  the  following  information:  interface  used  by  the  crew  member 
to  perform  the  tasks,  the  mental  resources  required  to  perform  the  task,  the  amount  of  the 
resource  required,  based  on  the  7-point  scales  presented  in  Table  2,  and  conflict  values  for  the 
multiple  tasks  occurring  simultaneously.  Based  on  these  values,  the  IMPRINT  software 
calculated  workload  predictions  for  the  operators  as  they  perform  the  tasks  associated  with  each 
function  in  each  mission.  The  researchers  then  examined  these  workload  predictions  to  see  if 
they  exceeded  acceptable  workload  levels. 
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3.  Analysis  and  Discussion 


After  the  ARL  researchers  completed  the  simulation  runs  for  the  scout  and  tank  models,  they 
analyzed  the  workload  data.  The  data  were  analyzed  for  the  button-push-level  models  only.  The 
button-push-level  models  were  VTT  Scout  Alexandria  and  VTT  Tank  Alexandria.  The  no- 
button-push  models,  VTT  Tank  and  VTT  Scout,  contain  similar  tasks  as  the  button-push  models, 
VTT  Tank  Alexandria  and  VTT  Scout  Alexandria.  The  tasks  were  decomposed  into  greater 
detail  in  the  button-push  models.  Because  the  difference  in  models  was  simply  attributable  to 
level  of  detail,  the  analysis  results  for  the  no-button-push  VTT  Tank  model  and  the  button-push 
VTT  Tank  Alexandria  would  be  the  same.  However,  in  the  VTT  Tank  Alexandria,  the  analysis 
would  tell  exactly  what  button  pushes  contributed  to  the  workload.  This  is  also  true  for  the  VTT 
Scout  and  VTT  Scout  Alexandria  models.  Therefore,  the  button-push  VTT  Tank  Alexandria  and 
VTT  Scout  Alexandria  were  the  models  runs  analyzed. 

The  first  step  in  analyzing  the  workload  data  is  to  determine  a  workload  threshold  for  the  data. 
The  workload  threshold  is  the  point  where  high  workload  is  expected  to  create  a  decrement  in 
performance.  Initially,  ARL  researchers  selected  a  workload  threshold  of  40  ±10  as  a  basis  from 
which  an  evaluation  of  workload  overload  would  be  performed.  They  selected  this  value 
because  previous  research  (Reid  and  Colle  1988)  had  determined  40  ±10  to  be  the  critical 
workload  values  for  predicting  operator  overload  in  an  air  crew  study.  However,  the  tasks  in  the 
Reid  and  Colle  study  were  flying  tasks,  and  the  tasks  in  the  VTT  are  ground  vehicle  tasks.  In 
addition,  the  Reid  and  Colle  workload  ratings  were  obtained  from  a  subjective  workload  scale 
rather  than  the  McCracken  and  Aldrich  scales  used  in  IMPRINT. 

Considering  these  differences,  when  ARL  and  MA&D  SMEs  reviewed  the  workload  peaks  from 
the  model  runs,  they  determined  that  this  was  too  low  a  threshold  for  the  VTT  tasks.  The  tasks 
performed  by  the  subjects  when  the  workload  was  in  the  30  to  50  range  could  most  likely  be 
performed  simultaneously  without  performance  decrement.  However,  a  review  of  the  tasks 
occurring  when  the  workload  was  60  or  above  indicated  that  performance  decrements  might 
occur  in  this  range.  Therefore,  the  researchers  selected  60  as  workload  threshold  for  this  study. 
The  following  sections  detail  each  model  simulation  run  and  the  tasks  that  the  operators  were 
performing  when  they  experienced  a  state  of  overload. 

3.1  Condition  1.  Scout  Alexandria  -  Platoon  Leader  (PL)  Driving  and  Scanning  for 
Targets 

This  model  scenario  was  designed  to  emulate  a  scout  mission  being  conducted  in  the  VTT 
simulator  and  allows  assignment  of  operator  tasks  performed  versus  automated  tasks.  The  two 
crew  members  performing  the  scout  mission  were  a  platoon  leader  and  a  sergeant.  In  this 
instance,  the  platoon  leader  was  assigned  the  driving  and  scanning  for  target  tasks  with  no 
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automation  aids  for  the  tasks.  The  model  represented  the  tasks  necessary  to  accomplish  the 
scenario  objectives  of  route  reconnaissance,  tactical  planning,  moving  to  and  occupying 
objective  position.  These  scenario  objective  functions  are  supplemented  by  the  operator 
functionality  necessary  to  successfully  accomplish  these  objectives.  Tasks  included  among  the 
operator  functionality  built  into  the  model  account  for  operator  actions  such  as  monitoring 
command  and  control  (C2),  platoon  leader  scanning  for  targets,  platoon  leader  driving,  sergeant 
and  platoon  leader  communicating,  and  a  variety  of  other  functions.  For  the  purposes  of  the  first 
scout  Alexandria  condition,  the  sergeant  was  eliminated  from  the  driving  tasks  because  the 
platoon  leader  was  the  driver.  However,  the  sergeant  did  scan  for  targets  and  communicate  with 
the  platoon  leader  via  a  headset. 

The  model  runs  showed  that  the  platoon  leader  experienced  109  instances  of  overload  over  the 
course  of  a  3-hour  mission  scenario  while  performing  scanning  and  driving  tasks.  The  tasks  that 
occurred  most  frequently  during  the  instances  of  overload  were  the  following: 

PL  glances  at  indirect  vision  display 
PL  scans  for  targets  with  thumb  on  control 
PL  adjusts  steering  using  handle 
PL  press  accelerator 
PL  talks  via  headset. 

The  highest  instance  of  operator  overloading  had  a  total  workload  score  of  199.16.  The  tasks 
that  were  performed  at  this  time  were 

PL  presses  accelerator 
PL  scans  for  targets  thumb  on  control 
PL  adjusts  steering  using  handle 
PL  glances  at  indirect  vision  display 
PL  listens  via  headset 

Together,  it  would  seem  that  these  tasks  should  not  be  overly  taxing  on  the  operator,  with  a 
single  task  demand  score  of  40.9,  but  the  inter-channel  conflict  score  was  a  152.18,  resulting  in 
the  extreme  total  workload  score.  There  are  also  inter-channel  demands  caused  by  the  number  of 
simultaneous  tasks  scheduled.  This  means  that  each  task  being  performed  here  instigated  much 
conflict  between  the  resources  required  to  perform  the  task.  Clearly,  the  score  occurred  as  a 
result  of  conflict  between  visual  resources  and  motor  resources,  noting  that  the  overloaded  tasks 
stated  before  indicate  a  great  deal  of  demand  placed  on  these  resources.  There  were  multiple 
visual  demands  for  scanning  and  monitoring  situational  awareness  via  the  indirect  vision  display 
pushing  the  inter-task  demand  value  beyond  limits. 
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3.2  Condition  2.  Scout  Alexandria  -  Sergeant  Driving  and  Scanning  for  Targets 

This  scenario  was  run  during  the  exact  same  conditions  as  condition  1 ,  with  the  exception  that 
the  sergeant  replaced  the  platoon  leader  in  all  driving  and  scanning  for  target  tasks.  In  this 
scenario,  the  sergeant  experienced  41  instances  of  overloading.  The  sergeant’s  instances  of 
overload  were  fewer  than  those  experienced  by  the  platoon  leader  in  condition  1 .  The  reason  for 
this  discrepancy  is  that  the  platoon  leader  was  required  to  perform  a  greater  number  of  other 
tasks  outside  the  scope  of  driving  and  scanning  for  targets  (e.g.,  monitoring  C2  communications 
and  overriding  the  sergeant  s  weapon  control).  The  sergeant  was  required  to  perform  far  fewer 
of  these  peripheral  tasks  than  the  platoon  leader,  thus  the  fewer  instances  of  overloading. 

The  following  five  tasks  occurred  most  frequently  in  this  condition: 

Sergeant  scans  for  targets,  thumb  on  control 
Sergeant  presses  accelerator 
Sergeant  listens  via  headset 
Sergeant  glances  at  indirect  vision  display 
Sergeant  adjusts  steering  using  handle 

It  is  interesting  to  note  that  these  are  the  same  five  tasks  that  had  the  greatest  frequency  of 
occurrence  in  condition  1 .  The  highest  instance  of  operator  overloading  had  a  total  workload 
value  of  1 77.02.  The  tasks  being  performed  during  this  time  were 

Sergeant  listens  via  headset 

Sergeant  scans  for  targets,  thumb  on  control 

Sergeant  presses  accelerator 

Sergeant  glances  at  indirect  vision  display 

Sergeant  checks  instruments  on  drive  display 

Sergeant  adjusts  steering  using  handle 


Similar  to  condition  1,  these  tasks  combined  for  an  extremely  high  inter-channel  conflict  value  of 
134.24.  Also,  these  are  nearly  the  same  tasks  involved  in  the  highest  instance  of  operator  work¬ 
load  in  condition  1  with  the  exception  of  the  sergeant  checking  instruments  on  drive  display  task. 

3.3  Condition  3.  Tank  Alexandria  -  Tank  Commander  (TC)  Driving  and  Scanning  for 
Targets 

This  model  scenario  was  designed  to  emulate  a  two-person  crew  operating  a  tank  in  the  VTT 
simulator  and  allows  various  configurations  of  operator  task  assignments  and  task  automation  to 
be  exercised.  The  two  crew  members  performing  this  tank  mission  were  a  tank  commander  and 
a  gunner.  In  this  condition,  the  tank  commander  was  assigned  to  perform  driving  tasks  as  well  as 
scan  for  target  tasks  with  no  automation  aid  available.  The  objective  functions  represented  by 


14 


this  model  range  from  conduction  of  a  tactical  road  march,  performing  a  support  by  fire,  hasty 
occupation  of  a  battle  position  and  platoon  consolidation.  Further,  the  model  accounts  for 
operator  functional  tasks  such  as  tank  commander  monitoring  C2,  communication  with  the 
gunner,  and  tank  commander  override  gunner  weapon  control.  In  this  condition,  the  gunner  was 
eliminated  from  all  driving  and  scan  for  target  tasks.  The  model  produced  the  following  results: 

The  tank  commander  experienced  164  instances  of  overloading  in  this  condition  over  the 
course  of  a  3 -hour  scenario.  The  tasks  that  occurred  most  frequently  during  this  time  were 

TC  adjusts  steering  using  handle 
TC  glances  at  indirect  vision  display 
TC  presses  accelerator 
TC  scans  for  targets,  thumb  on  control 

The  highest  instance  of  operator  overloading  had  a  total  workload  value  of  193.87.  The  tasks 
that  were  occurring  during  this  instance  were 

TC  develops  plan  using  map  multifunction  display 
TC  scans  for  targets  thumb  on  control 
TC  monitors  main  menu  for  incoming  message 
TC  listens  via  headset 
TC  glances  at  indirect  vision  display 

A  high  inter-channel  conflict  score  provided  the  extreme  workload  in  this  case  once  again  with 
the  inter-channel  conflict  value  at  151.77. 

3.4  Condition  4.  Tank  Alexandria  -  Gunner  Driving  and  Scanning  for  Targets 

This  condition  was  run  during  the  exact  same  conditions  as  condition  3  with  the  exception  that 
the  gunner  performed  the  driving  tasks  instead  of  the  tank  commander.  In  this  condition,  the 
gunner  experienced  32  instances  of  overloading.  Again,  the  discrepancy  between  the  amount  of 
overloading  experienced  by  the  gunner  versus  the  tank  commander  is  attributable  to  the  number 
of  extra  peripheral  tasks  that  the  tank  commander  was  required  to  perform.  Some  of  these 
peripheral  tasks  include  TC  monitoring  C2  as  well  as  TC  overriding  gunner  weapon  control.  The 
tasks  that  occurred  most  frequently  during  this  time  were  the  following: 

Gunner  adjusts  steering  using  handle 
Gunner  presses  accelerator 
Gunner  glances  at  indirect  vision  display 
Gunner  scans  for  targets,  thumb  on  control 
Gunner  talks  via  headset 
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The  highest  instance  of  operator  overloading  occurred  at  time  10042.83  with  a  total  workload 
value  of  120.96.  The  following  tasks  were  occurring  simultaneously  during  this  instance  of 
overload: 

Gunner  talks  via  headset 

Gunner  adjusts  steering  using  handle 

Gunner  checks  instruments  on  drive  display 

Gunner  presses  accelerator 

Gunner  glances  at  indirect  vision  display 

Gunner  scans  for  targets,  thumb  on  control 

Inter-channel  conflict  provided  the  extremely  high  workload  in  this  instance,  with  a  value  of 
72.66.  Also,  these  tasks  are  nearly  the  same  as  the  tasks  that  caused  extreme  overloading  in 
condition  3,  with  the  exception  of  these  tasks: 

TC  develops  plan  using  map  multifunction  display 
TC  monitors  main  menu  for  incoming  message 

This  would  seem  to  indicate  that  “communicating  via  headset,  glancing  at  indirect  vision  display, 
and  scanning  for  targets,  thumb  on  control”  require  resources  that  interfere  greatly  with  other 
resources  the  operator  requires  to  accomplish  a  task.  Notably,  these  are  similar  tasks  that  are 
occurring  during  the  extreme  instances  of  workload  from  conditions  1  and  2. 


4.  Conclusions  and  Recommendations 


Based  on  the  IMPRINT  analysis  results,  the  crew  member  who  is  acting  as  the  commander  of  the 
tank  or  scout  vehicle  will  experience  high  workload  when  he  or  she  is  expected  to  use  the  sight 
to  scan  for  targets  while  he  or  she  is  also  driving  and  performing  tasks  associated  with 
commanding  the  vehicle.  Therefore,  the  commander’s  workload  could  be  alleviated  if  some  of 
the  driving  or  scanning  tasks  were  automated.  The  current  version  of  the  VTT  will  include  an 
autonomous  mobility  capability  and  automated  target  scanning  which  should  alleviate  the 
commander’s  workload  by  eliminating  the  need  to  scan  for  targets.  However,  the  commander 
will  need  to  be  able  to  determine  if  a  target  detected  by  the  automated  system  is  friend  or  foe. 
Therefore,  future  modeling  efforts  should  look  at  the  commander’s  ability  to  effectively  perform 
the  identification  task  while  he  or  she  is  performing  other  tasks. 

The  high  workload  peaks  experienced  by  the  commander  during  driving  and  scanning  also 
occurred  because  he  or  she  was  required  to  look  at  two  different  screens  or  use  two  different  sets 
of  controls  at  the  same  time.  Some  of  these  events  that  occur  in  the  model  may  not  occur  in  the 


16 


real  world.  The  discrepancy  between  the  real  world  and  the  model  occurs  because  IMPRINT 
does  not  attempt  to  reschedule  tasks  when  conflicts  occur.  It  is  a  task-loading  model,  not  an 
operator-loading  model.  If  the  operator  is  scheduled  to  perform  four  simultaneous  visual  tasks, 
IMPRINT  will  allow  all  four  tasks  to  occur  with  a  very  high  workload  value.  Two  things  can 
mitigate  the  predictions  made  by  this  form  of  modeling.  The  first  is  that  the  actual  human  will 
prioritize  which  tasks  he  or  she  attends  to  and  will  serialize  his  or  her  operation  as  time  allows. 
The  second  is  that  the  scheduling  of  these  events  is  based  on  a  Monte  Carlo  engine.  The  Monte 
Carlo  engine  actually  determines  what  tasks  will  be  performed  in  what  order.  This  randomness 
of  scheduling  represents  the  attention  demands  placed  on  the  operator,  regardless  whether  the 
operator  can  actually  attend  to  the  demand.  Therefore,  TACOM  SMEs  should  review  the 
workload  analysis  and  identify  the  points  where  the  tasks  would  be  occurring  simultaneously  and 
the  commander  could  not  perform  an  alternate  strategy.  These  points  should  then  be  analyzed  to 
determine  if  the  tasks  could  be  (1)  redesigned  to  reduce  workload,  (2)  automated  to  reduce 
operator  workload,  or  (3)  situated  differently  between  the  system  operators  to  reduce  workload. 

In  addition,  researchers  could  try  changing  the  modality  required  for  tasks  during  those  points 
when  workload  is  very  high.  For  example,  for  those  points  where  the  commanders  are  required 
to  perform  simultaneous  visual  tasks,  presenting  some  of  the  visual  information  in  a  different 
mode  might  reduce  their  workload  by  reducing  conflict  within  the  visual  channel.  For  example, 
the  C2  messages  might  be  voice  activated  rather  than  visual  button  pushes.  This  change  is  a 
proposed  modification  of  the  VTT. 

Furthermore,  the  workload  associated  with  driving  tasks  for  both  the  commander  and  the  gunner 
might  be  reduced  if  they  share  driving.  The  VTT  does  have  the  capability  for  the  driving  to 
switch  back  and  forth  between  crew  members.  Future  models  or  testing  should  look  at  the 
effects  on  workload,  situational  awareness,  and  performance  for  the  crew  members  when  the 
switching  occurs. 

The  ARL  researchers  were  asked  to  build  the  simulator  model  using  the  scenario  that  TACOM 
researchers  used  in  their  Grayling  test.  This  TACOM  scenario  is  very  scripted.  Therefore,  the 
tasks  in  the  model  are  very  serial  and  are  executed  in  a  known  sequence  with  little  or  no  variation 
from  the  expected  course  of  execution.  Unforeseen  events  and  variable  states  of  the  world  (e.g., 
voice  communications  from  headquarters,  unexpected  threats)  are  not  accounted  for;  thus, 
workload  would  remain  consistent  across  multiple  runs  of  the  model.  A  more  dynamic  series  in 
the  model  and  in  the  Grayling  scenarios  would  provide  a  more  realistic  measure  of  operator 
workload.  In  addition,  this  dynamic  scenario  would  allow  the  model  tasks  and  workload  values 
to  be  validated.  Workload  values,  task  times,  and  task  accuracies  could  be  collected  from  two 
crew  members  as  they  perform  the  concurrent  tasks  in  the  dynamic  scenario.  These  data  would 
then  be  added  to  the  models,  and  further  analyses  could  be  conducted  with  the  validated  models. 
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